MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

专利摘要:
The present invention relates to a mobile terminal comprising a camera (121); a touch screen (151) for editing a received photo via the camera (121), said photo including at least a first image text portion having a first attribute and a second image text portion; with a second attribute different from the first attribute; and a controller (180) arranged to display first and second graphic objects (320a, 320b) indicating that the first and second image text portions can be selected, performing a first predetermined function corresponding to the first attribute of the first image text part when selecting the first graphic object or the first image text part, and executing a second predetermined function corresponding to the second attribute of the second image text part when selecting the second object graphic or the second part forming image text.
公开号:FR3041785A1
申请号:FR1654270
申请日:2016-05-12
公开日:2017-03-31
发明作者:Hyerim Ku；Yoomee Song；Arim Kwon；Hyungtae Jang；Jungmin Park；Hyungsun Kim；Kyuhee Kim
申请人:LG Electronics Inc；
IPC主号:

专利说明:

MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME
The present invention relates to a mobile terminal having a photo capture function and a method of controlling said mobile terminal.
In recent years, a function of reading or optical character recognition (OCR) has been applied to a mobile terminal equipped with a camera. More concretely, the OCR function reads characters (a text) using light. In the mobile terminal, the OCR function transforms into light signals the light reflected by letters, symbols and numbers printed or handwritten on paper.
In addition, the OCR function corresponds to an image to text conversion function. Indeed, the OCR function converts an image text portion included in the received photo through the camera into text. However, the OCR function of the mobile terminal has a limited character and a user-friendly mode of use.
Therefore, a first aspect of the detailed description is to provide a mobile terminal having an optimized user interface that uses a photo received through a camera, as well as a method of controlling said mobile terminal.
Another aspect of the detailed description is providing a mobile terminal for performing an optimized function using screen information included in a photo received through a camera, and a method of controlling said mobile terminal.
To provide the above and other advantages, and in accordance with the objectives of this specification as practiced and generally described herein, the present invention provides a mobile terminal comprising a camera; a touch screen adapted to edit a photo received through the camera, the photo having at least a first image text portion having a first attribute and a second image text portion having a second attribute different from the first attribute; and a controller adapted to display first and second graphic objects indicating that the first and second image text portions can be selected, performing a first predetermined function corresponding to the first attribute of the first image text portion when selecting the first graphic object or the first image text part, and executing a second predetermined function corresponding to the second attribute of the second image text part when selecting the second graphic object or the second text forming part. 'picture. The present invention also provides a corresponding method of controlling the mobile terminal. Other fields of application of the present application will emerge more clearly on reading the detailed description given below. However, it should be pointed out that the detailed description and the specific examples, although indicating preferred embodiments of the invention, are provided for purely illustrative purposes since, as will be apparent to those skilled in the art upon reading the Detailed description, it is possible to consider various variants and modifications thereof while remaining within the spirit and scope of the invention.
The accompanying drawings, which have been included to allow a better understanding of the invention and are an integral part of this specification, illustrate representative embodiments and serve, together with the description, to explain the principles of the invention.
In the drawings: Fig. 1A is a diagram showing a mobile terminal according to an embodiment of the present invention; Figs. 1B and 1C are conceptual diagrams representing the mobile terminal according to an embodiment of the present invention viewed from different directions; Fig. 2 is a flowchart showing a control method according to an embodiment of the present invention; Figs. 3A to 3C are diagrams showing the control method of Fig. 2; Figs. 4A-4C are conceptual diagrams representing a control method according to an embodiment of the present invention; and Figs. 5A-7B and Fig. 8 are conceptual drawings showing a control method according to another embodiment of the present invention.
We will now proceed to a detailed description, according to the representative embodiments disclosed herein and in light of the accompanying drawings. To abbreviate the description made with reference to the drawings, identical or equivalent components may be indicated by identical or similar numerical references, and their description is not repeated. In general, a suffix such as "module" and "unit" may be used to designate elements or components. The purpose of using such a suffix in this document is to facilitate the description of the booklet, since the suffix itself is not intended to constitute a special meaning or function. The accompanying drawings make it easy to understand various technical features and, as will be understood, the embodiments presented herein are in no way limited by the accompanying drawings. As such, the present invention is intended to encompass all modifications, equivalents and substitutes in addition to those specifically indicated in the accompanying drawings.
Although the terms first, second and so on may be used in this document to describe various elements, these terms are not intended to limit said elements. In general, these terms are used only to distinguish one element from another. When an element is referred to as "connected to" another element, that element may be connected to the other element but intermediate elements may also be present. On the other hand, when an element is qualified as "directly connected" to another element, no intermediate element is present.
A simple representation may include multiple representations unless it corresponds to a meaning that is clearly different from the context. We use terms such as "includes" or "features" in this document, and it should be understood that they serve to indicate the existence of several components, functions or steps described in the booklet, and we must also see a greater or lesser number of components, functions or steps can also be used.
The mobile terminals presented here can be implemented as terminals of various types and varied. Examples of these terminals include cell phones, smartphones or "smartphones", user devices, laptops, digital broadcast terminals, personal digital assistants (PDAs), portable media players, browsers , digital slates, digital tablets, ultrabooks, worn devices (eg, smart watches, smart glasses, virtual reality headsets) and the like. As an example of strictly non-limiting nature, we will deepen our description by referring to particular types of mobile terminals. However, these explanations also apply to other types of terminals, such as the types mentioned above. In addition, these explanations may also apply to fixed terminals such as digital TVs, desktops and the like.
Let us now examine Figures IA to IC, Figure IA being a block diagram of a mobile terminal according to the present invention, and Figures IB and IC of the conceptual views of a first example of the mobile terminal, seen from different directions. The mobile terminal 100 is shown having components such as a wireless communication unit 110, an input unit 120, a detection unit 140, an output unit 150, an interface unit 160, a memory 170, a control device 180 and a power supply unit 190. It is not mandatory to implement all the components shown, the components used may also be more or less numerous.
Now, consider Figure 1, in which the mobile terminal 100 is shown having a wireless communication unit 110 designed with several components commonly implemented. For example, the wireless communication unit 110 typically includes one or more components that allow wireless communication between the mobile terminal 100 and a wireless communication system or network in which the mobile terminal is located. The wireless communication unit 110 generally comprises one or more modules that allow communications such as wireless communications between the mobile terminal 100 and a wireless communication system, communications between the mobile terminal 100 and another mobile terminal, communications between the mobile terminal 100 and an external server. In addition, the wireless communication unit 110 generally includes one or more modules that connect the mobile terminal 100 to one or more networks. To facilitate these communications, the wireless communication unit 110 comprises one or more of the following elements: a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. The input unit 120 includes a camera 121 which provides images or video, a microphone 121 which is a type of audio input inputting an audio signal; and a user input unit 122 (for example, a touch key, a key, a mechanical key, a function key, and the like) that allows a user to enter information. Data (e.g., audio, video, image and the like) is obtained by the input unit 120 and can be analyzed and processed by the controller 180 according to device parameters, user commands or combinations of both. The detection unit 140 is generally implemented by means of one or more sensors designed to detect the internal information of the mobile terminal, the ambient environment of the mobile terminal, user information and the like. For example, in FIG. 1A, the detection unit 140 is shown provided with a proximity sensor 141 and a lighting sensor 142.
Where appropriate, the detection unit 140 may comprise, alternatively or in addition, other types of sensors or devices such as a touch sensor, an acceleration sensor, a magnetic sensor, an accelerometer, a sensor a gyroscope, a motion sensor, an RGB sensor, an infrared (IR) sensor, a fingerprint sensor, an ultrasonic sensor, an optical sensor (for example, the camera 121), a microphone 121, a battery meter, a room sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detector, a thermal sensor and a gas sensor, among others), and a chemical sensor (for example, an electronic nose, a medical sensor, biometric sensor and the like), to name a few. The mobile terminal 100 may be designed to use the information obtained from the detection unit 140 and, in particular, information obtained from one or more sensors of the detection unit 140, and combinations of these different information. The output unit 150 is generally designed to edit various types of information, including audio, video, tactile and similar outputs. The output unit 150 is shown having a display unit 151, an audio output module 152, a haptic module 153 and an optical output module 154. The display unit 151 can have a layered structure or integrated structure with a touch sensor that simplifies the touch screen. The touch screen may constitute an output interface between the mobile terminal 100 and a user, and it may also function as a user input unit 122 which provides the input interface between the mobile terminal 100 and the user. user. The interface unit 160 serves as an interface with different types of external devices that can be coupled to the mobile terminal 100. The interface unit 160 may include, for example, one or more of the following: wired or non-wired ports wire, external power ports, wired or wireless data ports, memory card ports, connection ports to a device with an identification module, input / output ports (I / O) audio, video I / O ports, headphone ports, and the like. In some cases, the mobile terminal 100 may perform various control functions associated with a connected external device, when the external device is connected to the interface unit 160.
The memory 170 is generally implemented in such a way as to store data enabling the various functions or characteristics of the mobile terminal 100 to be supported. For example, the memory 170 may be designed to store application programs executed in the mobile terminal 100. data or instructions relating to the operations of the mobile terminal 100 and the like. Some of these application programs can be downloaded from an external server via wireless communication. Other application programs may be installed in the mobile terminal 100 during its manufacture or its transport, which is generally the case for the basic functions of the mobile terminal 100 (for example, the reception of a call, the making a call, receiving a message, sending a message and the like). It is common for application programs to be stored in the memory 170, installed on the mobile terminal 100 and executed by the controller 180 to perform an operation (or perform a function) for the mobile terminal 100.
The control device 180 generally has the function of controlling the overall operation of the mobile terminal 100, in addition to the operations associated with the application programs. The controller 180 may provide or process user-adapted information or functions by processing the signals, data, information, and the like that are input or output of the various components shown in Figure 1, or by enabling application programs stored in the memory 170. By way of example, the control device 180 controls all or part of the components shown in FIG. 1 according to the execution of an application program that has been stored in the memory 170. The power supply unit 190 may be adapted to receive external energy or provide internal energy to provide the energy required for the operation of the elements and components included in the mobile terminal 100. The power supply unit 190 may comprise a battery , the battery may be designed to be integrated into the body of the terminal or designed to be separate from the body of the terminal nal.
At least some of the aforementioned components may operate cooperatively to implement a control operation or method of a glass type terminal according to various embodiments which will be explained later. The operation or control method of the glass type terminal can be implemented on the glass type terminal by controlling at least one application program stored in the memory 170.
Still with reference to FIG. 1A, we will now detail various components shown in this figure. With respect to the wireless communication unit 110, the broadcast receiving module 111 is generally adapted to receive a broadcast signal and / or information associated with a broadcast from an external radio-broadcast management entity. through a broadcast channel. The broadcast channel may include a satellite channel, a terrestrial channel, or both. In some embodiments, two or more broadcast receiving modules 111 may be used to facilitate the simultaneous reception of two or more broadcast channels, or to allow switching between different broadcast channels.
The mobile communication module 112 may transmit and / or receive wireless signals from or to one or more network entities. Typical examples of a network entity include a base station, an external mobile terminal, a server, and the like. These network entities are part of a mobile communication network that has been established in accordance with technical standards or communication methods for mobile communications (eg the global mobile communications system (GSM), (CDMA), CDMA2000 (code division multiple access 2000), EV-DO (optimized enhanced voice-data communication or voice-data communication only), broadband CDMA (WCDMA), access broadband downlink packets (HSDPA), high-speed uplink packet access (HSUPA), high-speed uplink packet access, long-term evolution (LTE), Long-Term Advanced (LTE-A) and the like As examples of wireless signals transmitted and / or received via the mobile communication module 112 are audio call signals, video call signals (telephony), or various fo data files that support text or multimedia message communications.
The wireless Internet module 113 is designed to facilitate wireless access to the Internet. This module can be internally or externally coupled to the mobile terminal 100. The wireless Internet module 113 can transmit and / or receive wireless signals via communication networks using wireless Internet technologies.
Examples of this wireless Internet access include wireless local area network (WLAN), Wi-Fi, Wi-Fi Direct, DLNA, wireless broadband (WiBro), international interoperability for wireless Internet access. Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Evolution long-term advanced (LTE-A) and the like. The wireless Internet module 113 may transmit or receive data through one or more of these wireless Internet technologies, as well as other Internet technologies.
In some embodiments, when wireless Internet access is implemented according to, for example, WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A protocols and the like, as part of a mobile communication network, it is the wireless Internet module 113 that ensures this access to wireless Internet. As such, the Internet module 113 can cooperate with the mobile communication module 112 or fulfill the function of the latter.
The short-range communication module 114 is designed to facilitate short-range communications. Technologies for implementing these short-range communications include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, near field communications (NFC), Wi-Fi, Wi-Fi Direct, wireless USB and the like. The short-range communication module 114 generally supports wireless communications between the mobile terminal 100 and a wireless communication system, communications between the mobile terminal 100 and another mobile terminal 100, or communications between the mobile terminal. and a network where there is another mobile terminal 100 (or an external server), via wireless local area networks. An example of wireless LANs is wireless personal networks.
In some embodiments, another mobile terminal (which may be similarly designed to the mobile terminal 100) may be a worn device, for example, a smart watch, smart glasses or a virtual reality headset, which can exchange data with the mobile terminal 100 (or cooperate in another way with the mobile terminal 100). The short-range communication module 114 can detect or recognize the ported device, and allow communication between the ported device and the mobile terminal 100. In addition, when the detected ported device is a device that has been authenticated to communicate with the terminal mobile 100, the control device 180 can make, for example, that the data processed in the mobile terminal 100 are transmitted to the device carried via the short-range communication module 114.
Thus, a user of the worn device can use the processed data in the mobile terminal 100 on the device carried. For example, when a call is received in the mobile terminal 100, the user can answer the call using the ported device. Similarly, when a message is received in the mobile terminal 100, the user can view the received message using the worn device.
The location information module 115 is generally designed to detect, calculate, obtain or otherwise identify the position of the mobile terminal. For example, the location information module 115 includes a global positioning system (GPS) module, a Wi-Fi module, or both. If desired, the location information module 115 may alternatively or additionally work with any of the other modules of the wireless communication unit 110 to obtain data relating to the position of the mobile terminal. For example, when the mobile terminal uses a GPS module, the position of the mobile terminal can be acquired through a signal sent from a GPS satellite. As another example, when the mobile terminal uses the Wi-Fi module, the position of the mobile terminal can be acquired from information relating to a wireless access point that transmits or receives a wireless signal to or from the module. Wi-Fi. The input unit 120 may be designed to allow different types of input to the mobile terminal 120. Examples of these inputs are sound, picture, video, data, and a video. user input. An image and video type input is often obtained by means of one or more cameras 121. These cameras 121 can process frames of still image or video images obtained by image sensors in video capture mode. photo capture. The processed image frames may be displayed on the display unit 151 or stored in the memory 170. In some cases, the cameras 121 may be arranged in a matrix configuration to allow a plurality of images having Various angles or foci are entered into the mobile terminal 100. As another example, the cameras 121 may be placed in a stereoscopic arrangement to acquire left and right images for implementing a stereoscopic image.
The microphone 122 is generally implemented so as to allow audio input into the mobile terminal 100. The audio input can be processed in various ways depending on the function performed in the mobile terminal 100. Where appropriate, the microphone 122 may include different noise suppression algorithms to eliminate unwanted noise produced during the reception of external audio signals. The user input unit 123 is a component that allows a user to make an entry. This user input may enable the controller 180 to control the operation of the mobile terminal 100. The user input unit 123 may include one or more of the following elements: a mechanical input element (for example, a key, a button located on a front and / or rear surface or on a lateral surface of the mobile terminal 100, a dome-type switch, an impulse control wheel, an impulse control switch and the like) or a touch input, among other. For example, the touch input may be a virtual key or a function key, which is displayed on a touch screen by software processing, or a touch key that is located at a location of the mobile terminal other than the touch screen. In addition, the virtual key or the visual key may be displayed on the touch screen in various forms, for example, a graphic, text, an icon, video, or a combination of these different forms. The detection unit 140 is generally designed to detect one or more of the following information: internal information of the mobile terminal, information on the ambient environment of the mobile terminal, user information, or the like. The control device 180 generally cooperates with the transmitter unit 140 to control the operation of the mobile terminal 100 or execute a data processing, a function or an operation associated with an application program installed in the mobile terminal from the detection provided by the detection unit 140. The detection unit 140 may be implemented by any of a variety of sensors, some of which will now be described in more detail.
The proximity sensor 141 may comprise a sensor for detecting the presence or absence of an object approaching a surface, or an object located near a surface, using an electromagnetic field, radii infrared or the like without mechanical contact. The proximity sensor 141 may be arranged in an internal region of the mobile terminal covered by the touch screen, or near the touch screen.
The proximity sensor 141 may include, for example, any of the following sensors: a transmissive type photoelectric sensor, a direct reflective type photoelectric sensor, a reflective mirror type photoelectric sensor, a high frequency oscillation proximity sensor , a capacitance type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like. When the touch screen implemented is of the capacitance type, the proximity sensor 141 can detect the proximity of a pointer relative to the touch screen due to the variations of an electromagnetic field that reacts to the approach of an object with conductivity. In this example, the touch screen (touch sensor) can also be classified as a proximity sensor.
In this document, we will often use the term "touch close" to describe the scenario in which the pointer is placed so as to be close to the touch screen without coming into contact with it. In this paper, we will often use the term "touch-to-touch" to describe the scenario in which a pointer comes into physical contact with the touch screen. For the position corresponding to the touch near the pointer relative to the touch screen, this position corresponds to a position where the pointer is perpendicular to the touch screen. The proximity sensor 141 can detect a nearby touch, and touch profiles in proximity (eg, distance, direction, speed, time, position, motion status, and the like).
In general, the controller 180 processes the data that corresponds to the nearby touches and proximity touch profiles detected by the proximity sensor 141, and triggers the editing of visual information on the touch screen. In addition, the controller 180 may control the mobile terminal 100 to perform different operations or to process different data depending on whether a touch relative to a point of the touch screen is a touch in the vicinity or a touch contact.
A touch sensor may detect a touch applied to the touch screen, for example to the display unit 151, using any of a variety of touch methods. Examples of such tactile methods include a resistive type, capacitive type, infrared type and magnetic field type, among others. By way of example, the touch sensor may be designed to convert pressure variations applied to a specific part of the display unit 151, or to convert a capacitance appearing in a specific part of the display unit 151. , as electrical input signals. The touch sensor may also be designed to detect not only a touched location and an affected area, but also tactile pressure and / or touch capability. To apply a touch input to the touch sensor, a touch object is generally used. Examples of typical touch objects are a finger, a touch pen, a stylus, a pointer, or the like.
When a touch input is detected by a touch sensor, corresponding signals can be transmitted to a touch controller. The touch controller can process the received signals and then transmit the data corresponding to the controller 180. Accordingly, the controller 180 can detect the region of the display unit 151 that has been touched. Here, the touch control device may be a separate component of the controller 180, the controller 180, or a combination of both.
In some embodiments, the controller 180 may execute the same commands or different commands depending on the type of touch object that touches the touch screen or a key provided in addition to the touch screen. The execution of the same commands or different commands depending on the object that provides a touch input can be decided, for example, depending on the current state of operation of the mobile terminal 100 or a running application program .
The touch sensor and the proximity sensor can be implemented individually, or combined, to detect different types of touch. These touches include a short touch (or tapping), a long touch, multiple touch, a drag-and-drop feel, a scroll-like touch, a zoom-in feel, a zoom-out feel, a touch of drag type, hover type touch and the like.
Where appropriate, an ultrasonic sensor may be implemented to recognize positional information relating to a touch object by means of ultrasonic waves. The controller 180 may, for example, calculate the position of a wave generation source based on information detected by a lighting sensor and a plurality of ultrasonic sensors. Light being much faster than ultrasonic waves, the time taken by the light to reach the optical sensor is much shorter than the time taken by the ultrasonic wave to reach the ultrasonic sensor. The position of the wave generation source can be calculated by this fact. For example, the position of the wave generation source can be calculated as a function of the time offset from when the ultrasonic wave reaches the sensor, the light serving as a reference signal.
The camera 121 generally comprises at least one of a camera sensor (CCD, CMOS etc.), a photoelectric sensor (or image sensors), or a laser sensor. The implementation of the camera 121 with a laser sensor can allow the detection of the touch of a physical object with respect to a stereoscopic 3D image. The photoelectric sensor can be superimposed on the display device or cover it. The photoelectric sensor can be designed to scan the movement of the physical object near the touch screen. If we go into the details, the photoelectric sensor may include photodiodes and row and column transistors for scanning content received in the photoelectric sensor by means of an electrical signal that varies depending on the amount of light applied. Indeed, the photoelectric sensor can calculate the coordinates of the physical object according to light variations to obtain information on the position of the physical object. The display unit 151 is generally designed to edit processed information in the mobile terminal 100. For example, the display unit 151 may display information of the execution screen of an application program executing on the mobile terminal 100 or user interface (UI) and graphical user interface (GUI) information in response to the information of the execution screen. In some embodiments, the display unit 151 may be implemented as a stereoscopic display unit for displaying stereoscopic images. A typical stereoscopic display unit may employ a stereoscopic display system such as a stereoscopic system (glass system), an auto-stereoscopic system (glassless system), a projection system (holographic system) or the like.
In general, a 3D stereoscopic image may include a left image (e.g., an image of the left eye) and a right image (e.g., an image of the right eye). Depending on how the left and right images combine to form a 3D stereoscopic image, a 3D stereoscopic imaging process can be divided into a top-down process in which the left and right images are at the top and bottom of the image. a frame, a process from left to right (from left to right or side by side) in which the left and right images are to the left and to the right of a frame, a checker-like process in which fragments of the left images and right are in the form of mosaic, an interleaving method in which the left and right images are arranged in turn by column or rows, and a chronological method (or frame by frame) in which the left and right images are alternately display on a chronological basis.
In addition, as for a 3D thumbnail, a left thumbnail and a right thumbnail can be generated from a left image and a right image of an initial image frame, respectively, and then combine to produce a only 3D vignette. In general, the term "vignette" may be used to designate a reduced image or a reduced still image. The left thumbnail and the right thumbnail generated can be displayed separated by a horizontal distance whose depth corresponds to the disparity between the left image and the right image on the screen, which gives a sensation of stereoscopic space. The left image and the right image necessary to implement a 3D stereoscopic image can be displayed on the stereoscopic display unit through a stereoscopic processing unit. The stereoscopic processing unit can receive the 3D image and extract the left image and the right image, or can receive the 2D image and transform it into a left image and a right image.
The audio output module 152 is generally designed to output audio data. This audio data can come from any of several different sources, so that the audio data can be received from the wireless communication unit 110 or may have been stored in the memory 170. The audio data can be transmitted in modes such as a signal receiving mode, a calling mode, a recording mode, a speech recognition mode, a broadcasting receiving mode and the like. The audio output module 152 may provide an audible output relating to a particular function (e.g., a call tone receiving tone, a message receiving tone, etc.) performed by the mobile terminal 100. The output module Audio 152 may also be implemented as a receiver, speaker, horn or the like.
A haptic module 153 may be designed to produce various tactile effects that a user feels, perceives or experiences in another way. A typical example of a tactile effect produced by the haptic module 153 is that of a vibration. The power, type and other characteristics of the vibration produced by the haptic module 153 can be controlled through a selection or a user parameter by the controller. For example, the haptic module 153 may emit different vibrations in a combined or sequential manner.
In addition to the vibrations, the haptic module 153 can produce various other tactile effects, including a stimulating effect such as a vertically movable hairpin arrangement for touching the skin, a spraying force, or an air suction force through a jet orifice or suction opening, a touch of the skin, an electrode contact, an electrostatic force, an effect that reproduces the sensation of cold or heat by using an element that can absorb or produce heat, and the like.
The haptic module 153 can also be implemented so as to allow the user to feel a tactile effect through a muscular sensation, for example at the user's fingers or arm, as well as transfer the tactile effect by direct contact. Two or more haptic modules 153 may be provided, depending on the particular configuration of the mobile terminal 100.
An optical output module 154 may emit a signal to indicate the creation of an event using light from a light source. Examples of events created in the mobile terminal 100 include receiving a message, receiving a call signal, a missed call, an alarm, a calendar notification, receiving an electronic message, receiving information through an application, and the like. A signal emitted by the optical output module 154 may be implemented such that the mobile terminal emits monochromatic light or light having a plurality of colors. The transmitted signal can end as soon as the mobile terminal detects that a user has consulted the created event, for example. The interface unit 160 serves as an interface for allowing external devices to be connected to the mobile terminal 100. For example, the interface unit 160 can receive data transmitted by an external device, receive power from transfer to the elements and components in the mobile terminal 100, or transmit internal data of the mobile terminal 100 to the external device in question. The interface unit 160 may comprise wired or wireless headphone ports, external power ports, wired or wireless data ports, memory card ports, connection ports to a device with an identification module, audio input / output (I / O) ports, video I / O ports, headphone ports, or the like.
The identification module may be a chip that stores various information for authenticating the use authority of the mobile terminal 100 and may include a user identity module (UIM), an identity module of subscriber (SIM), universal subscriber identity module (USIM) and the like. In addition, the device with the identification module (also called "identification device" in this document) may be in the form of a smart card. Therefore, the identification device can be connected to the terminal 100 via the interface unit 160.
When the mobile terminal 100 is connected to an external pedestal, the interface unit 160 can serve as a passageway to allow the power of the pedestal to power the mobile terminal 100 or can be used as a gateway to allow various control signals inputted by the user from the base are transferred to the mobile terminal through it. Various control signals or energy input from the base can serve as signals to recognize that the mobile terminal is properly attached to the base.
The memory 170 may store programs for supporting operations of the controller 180 and store input / output data (e.g. a directory, messages, still images, videos, etc.). The memory 170 can store data relating to various types of vibrations and audio that are transmitted in response to touch inputs on the touch screen.
The memory 170 may comprise one or more types of storage media, in particular a flash memory, a hard disk, an SSD disk, a silicon disk, a micro-type multimedia card, a card-type memory (for example an SD memory or DX, etc.), random access memory (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), memory magnetic disk, a magnetic disk, an optical disk and the like. The mobile terminal 100 may also be used as part of a network storage device that performs the storage function of the memory 170 on a network, for example the Internet.
The controller 180 may generally control the overall operation of the mobile terminal 100. For example, the controller 180 may enable or disable a lockout state to prevent a user from entering a control command with respect to applications. when the state of the mobile terminal fulfills a predefined condition. The controller 180 may also perform controls and processing associated with voice calls, data transmissions, video calls, and the like, or perform pattern recognition processing to recognize a handwritten input or a drawing input made to the device. touch screen as characters or as images, respectively. In addition, the controller 180 may control one or a combination of these components to implement various representative embodiments described herein. The power supply unit 190 receives external energy or provides internal energy and provides the appropriate amount of power required for the operation of the respective elements and components included in the mobile terminal 100. The power unit 190 can include a battery, which is generally rechargeable or can be removably coupled to the terminal body for charging. The power supply unit 190 may include a connection port. The connection port may be designed as an example of the interface unit 160 to which an external charger is electrically connected to provide power to recharge the battery.
As another example, the power supply unit 190 may be designed to recharge the battery in a non-wired manner, without using the connection port. In this example, the power supply unit 190 can receive power, transferred from an external wireless power transmitter, using at least one of an inductive coupling method based on magnetic induction, or a coupling method. magnetic resonance based on electromagnetic resonance. Various embodiments described herein may be implemented on a computer readable medium, a machine readable medium or a similar medium by, for example, software, hardware or any combination thereof. this.
Let us now examine FIGS. 1B and 1C, on which the mobile terminal 100 is described as a terminal body of the bar type. However, the mobile terminal 100 may also be implemented in any of a number of different configurations. Examples of such configurations include the watch type, the clip type, the goggle type, or the collapsible type, the flip type, the slide type, the tilt type, and the pivot type in which two bodies or more are combined with each other in a relatively mobile manner, and combinations thereof. The explanations provided in this document will often relate to a particular type of mobile terminal (for example, the bar type, the watch type, the glasses type, and the like). However, these explanations relating to a particular type of mobile terminal generally also apply to other types of mobile terminals.
The mobile terminal 100 generally comprises a housing (for example a frame, a housing, a cover and the like) which forms the appearance of the terminal. In this embodiment, the housing is formed using a front housing 101 and a rear housing 102. Various electronic components are integrated in a space formed between the front housing 101 and the rear housing 102. At least an intermediate casing may be placed further between the front casing 101 and the rear casing 102. The display unit 151 is shown located on the front face of the terminal body for editing information. As seen in the drawing, a window 151a of the display unit 151 may be mounted on the front housing 101 to form the front surface of the terminal body together with the front housing 101. In some embodiments, Electronic components may also be mounted on the rear case 102. Examples of such electronic components include a removable battery 191, an identification module, a memory card and the like. A back cover 103 is shown as covering the electronic components, and this cover can be detachably coupled to the rear case 102. Therefore, when the rear cover 103 is detached from the rear case 102, the electronic components mounted on the rear case 102 are exposed externally.
As we see in the drawing, when the rear cap 103 is coupled to the rear casing 102, a side surface of the rear casing 102 is partially exposed. In some cases, at the time of coupling, the rear case 102 may also be completely protected by the back cover 103. In some embodiments, the back cover 103 may include an opening for externally exposing a camera 121b or a display module. 152b audio output.
The casings 101, 102, 103 may be formed by injection molding of a synthetic resin or may be formed of a metal, for example stainless steel, aluminum (Al), titanium (Ti) or similar. As an alternative to the example in which the plurality of housings form an internal space for housing components, the mobile terminal 100 may be designed such that a single housing forms this internal space. In this example, a one-piece mobile terminal 100 is formed such that the synthetic resin or metal extends from a side surface to a back surface.
If necessary, the mobile terminal 100 may include a sealing unit that prevents the introduction of water into the body of the terminal. For example, the sealing unit may comprise a sealing element located between the window 151a and the front casing 101, between the front casing 101 and the rear casing 102, or between the rear casing 102 and the rear cap 103, so as to seal an internal space when these housings are coupled.
Figures IB and IC represent certain components arranged on the mobile terminal. However, other arrangements are possible within the scope of the teachings of the present invention. Some components may be deleted or rearranged. For example, the first handling unit 123a may be on another surface of the terminal body, and the second audio output module 152b may be on the side surface of the terminal body. The display unit 151 outputs processed information in the mobile terminal 100. The display unit 151 may be implemented by means of one or more suitable display devices. Examples of such suitable display devices include a liquid crystal display (LCD), a thin-film transistor (TFT-LCD) and liquid crystal display, an organic light-emitting diode (OLED), a flexible display, a three-dimensional screen (3D), an electronic ink display device, and combinations of these different devices. The display unit 151 may be implemented by means of two display devices that can implement the same or different display technologies. For example, a plurality of display units 151 may be arranged on one side, or separated from one another, or these devices may be integrated, or these devices may be arranged on different surfaces. The display unit 151 may also include a touch sensor that detects a touch input received in the display unit. When entering a touch in the display unit 151, the touch sensor may be designed to detect that touch and the controller 180 may, for example, issue a control command or other signal corresponding to the touch. Content entered by touch can be a text or numeric value, or a menu item that can be specified or designated in different modes.
The touch sensor may be designed as a touch-patterned film disposed between the window 151a and a display on a rear surface of the window 151a, or a patterned wire formed directly on the back surface of the window 151a. the window 151a. Alternatively, the touch sensor can be formed integrally with the display. For example, the touch sensor may be disposed on a substrate of the display or in the display. The display unit 151 may also form a touch screen together with the touch sensor. Here, the touch screen can serve as a user input unit 123 (see Figure IA). Thus, the touch screen can replace at least part of the functions of the first handling unit 123a. The first audio output module 152a may be implemented as a speaker for transmitting voice audio signals, audible alarms, multimedia audio reproduction and the like.
The window 151a of the display unit 151 generally includes an aperture for allowing the passage of audio signals produced by the first audio output module 152a. One possibility is to allow the broadcasting of audio signals along an assembly space between the structural bodies (for example, a gap between the window 151a and the front housing 101). In this case, an independently formed hole for emitting sounds may be invisible or have a hidden appearance, further simplifying the appearance and fabrication of the mobile terminal 100.
The optical output module 154 may be designed to emit light to indicate the creation of an event. Examples of such events include receiving a message, receiving a call signal, a missed call, an alarm, a calendar notification, receiving an e-mail message, receiving information at the same time. through an application and the like. When a user has viewed the created event, the controller may command the optical output unit 154 to terminate the light emission.
The first camera 121a can process frames of images such as still or moving images obtained by the image sensor in a capture mode or a video call mode. The processed image frames may then be displayed on the display unit 151 or stored in the memory 170. The first and second handling units 123a and 123b are examples of the user input unit 123 which may be manipulated by a user to provide an input to the mobile terminal 100. The first and second handling units 123a and 123b may also be commonly referred to as the handling portion, and may employ any tactile method that allows the user to perform such manipulation. than a touch, a pressure, a scroll or the like. The first and second handling units 123a and 123b may also employ any non-tactile method that allows the user to perform such manipulation as a nearby touch, an overflight, or the like.
Fig. 1B shows the first manipulation unit 123a as a touch key, but other options are possible, including a mechanical key, a key, a touch key, and combinations thereof. An input received in the first and second handling units 123a and 123b can be used in various ways. For example, the first handling unit 123a may be used by the user to provide a menu entry, a home key, a cancel, a search, or the like, and the second handling unit 123b may be used by the user. user to provide an input to control the sound volume delivered by the first or second audio output modules 152a or 152b, to switch to touch recognition mode of the display unit 151, or the like.
As another example of the user input unit 123, a rear input unit may be located on the rear surface of the terminal body. The rear input unit may be manipulated by a user to provide an input to the mobile terminal 100. The input may be used in different ways. For example, the rear input unit may be used by the user to provide an input to turn on or off, go to the beginning, go to the end, scroll, control the sound volume delivered by the first or second output modules 152a or 152b audio, switch to touch recognition mode of the display unit 151, and the like. The rear input unit can be designed to allow touch input, pressure input, or combinations of these inputs.
The rear input unit may be positioned to cover the display unit 151 on the front side in the thickness direction of the terminal body. For example, the rear input unit may be on a front end portion of the back side of the terminal body so that a user can easily manipulate it with his index finger when the user enters the body of the terminal. terminal of a hand. Alternatively, the rear input unit may be located virtually anywhere on the back side of the terminal body.
Embodiments that include the rear input unit may implement some or all of the functionality of the first handling unit 123a in the rear input unit. Thus, in cases where the first handling unit 123a has been removed from the front side, the display unit 151 may have a larger screen. As another possibility, the mobile terminal 100 may comprise a fingerprint sensor that scans the fingerprint of the user. The controller 180 can then use the fingerprint information detected by the fingerprint sensor as part of the authentication procedure. The fingerprint sensor may also be installed in the display unit 151 or implemented in the user input unit 123. The microphone 122 is shown located at one end of the mobile terminal 100, but other locations are possible. If necessary, several microphones can be implemented, such an arrangement allowing the reception of stereo sounds. The interface unit 160 may serve as a path for allowing the mobile terminal 100 to interface with external devices. For example, the interface unit 160 may comprise one or more of the following: a connection terminal for connecting to another device (for example, a listener, an external speaker, or the like), a port of near-field communication (for example, an infrared data association (IrDA) port, a Bluetooth port, a wireless LAN port, and the like), or a power terminal for powering the terminal The interface unit 160 may be implemented in the form of a socket for receiving an external card, such as a subscriber identification module (SIM), an identity module of user (UIM), or a memory card for storing information.
The second camera 121b is shown on the rear side of the terminal body and includes a photo capture direction substantially opposite to the photo capture direction of the first camera unit 121a. Where appropriate, the second camera 121a may also be placed at other locations, or designed to be movable, to have a different photo capture direction than that shown.
The second camera 121b may include a plurality of lenses arranged along at least one line. The plurality of lenses may also be arranged in a matrix configuration. Cameras can be called an "online camera". When the second camera 121b is implemented as an on-line camera, images can be captured in different ways using the plurality of lenses and the images are of better quality.
As we see in FIG. 1C, a flash 124 is shown next to the second camera 121b. When an image of a subject is captured with the camera 121b, the flash 124 may illuminate the subject. As we see in FIG. 1C, the second audio output module 152b may be on the body of the terminal. The second audio output module 152b may implement stereophonic sound functions in conjunction with the first audio output module 152a, and may also be used to implement a hands-free telephone mode for communications.
At least one antenna for wireless communications may be on the body of the terminal. This antenna can be installed in the body of the terminal or formed by the housing. For example, an antenna that forms part of the broadcast receiving module 111 may enter the body of the terminal. Alternatively, an antenna may be formed by means of a film attached to an inner surface of the back cover 103, or a housing that includes a conductive material.
A power supply unit 190 for powering the mobile terminal 100 may include a battery 191, which is mounted in the body of the terminal or releasably coupled to the outer portion of the terminal body. The battery 191 can receive power via a power cable connected to the interface unit 160. In addition, the battery 191 can be wirelessly recharged by means of a wireless charger. Wireless charging can be implemented by magnetic induction or electromagnetic resonance.
The back cover 103 is shown coupled to the rear case 102 to protect the battery 191, to prevent separation of the battery 191, and to protect the battery 191 against external shocks or foreign bodies. When the battery 191 can be detached from the terminal body, the rear case 103 can be detachably coupled to the rear case 102.
An accessory making it possible to protect the appearance of the mobile terminal 100 or to facilitate or to deliver its functions can also be provided on the mobile terminal 100. As an example of an accessory, a cover or a cover allowing to cover or to receive at least one surface of the mobile terminal 100 may be provided. The cover or the cover can cooperate with the display unit 151 to deliver the functions of the mobile terminal 100. Another example of this accessory is a touch pen for facilitating or delivering a touch input on a touch screen.
A mobile terminal 100 according to an embodiment of the present invention can store a photo received through the camera 121 in the memory 170. In addition, the storage of the photo in the memory 170 is expressed in terms of "capture photo of a photographic subject "" capture of a photo "," acquisition of a photo "," realization of a photo capture operation ", and so on. However, no limit is imposed on these expressions, and any expression can be used to indicate that the picture received through the camera 121 is stored in the memory 170.
Further, the execution of the photo capture function corresponds to the execution of the photo capture application, and when the photo capture function is performed, the controller 180 activates the camera 121 and controls to the camera 122 to capture photos. In addition, in one example, the photo capture function is performed after selecting (or touching) an icon associated with the photo capture function. When the photo capture function is executed after selecting this icon, the display unit 151 is activated. Furthermore, in addition to the method of selecting an icon associated with the photo capture function, when selecting at least one key (for example, at least one of the hardware keys or a softkey) among the keys which are provided on the mobile terminal, the photo capture function is executed. In this case, although the display unit 151 is turned off, the controller 180 may perform the photo capture function when selecting the key provided on the mobile terminal.
Further, the mobile terminal according to an embodiment of the present invention performs the photo capture operation according to a selection of the user, and the selection of the user is expressed in the terms "control of user control "or" control command ". In addition, the selection of the user is done in various ways. For example, a user performs the photo capture operation by touching or pressing the hardware key provided on the mobile, or by touching the function key or the visual key appearing on the display unit 151. Indeed, when this touch or pressure is applied to the hardware key associated with the photo capture function or when this touch is applied to the function key or the visual key appearing on the display unit 151, the controller 180 of the mobile terminal 100 determines that a user control command for performing the photo capture operation has been received.
In addition, and in a nonlimiting manner, the mobile terminal 100 according to one embodiment of the present invention captures a photo in various ways after receiving the user control command. For example, when the received photo (input) through the camera 121 is edited on the display unit 151, the controller 180 captures the photo after applying a touch of a predetermined type to the photo or the display unit 151.
The touch of a predetermined type is a touch associated with the photo capture operation and includes different types of touch. For example, the touches of a predetermined type include different types of touch such as a short touch (tapping), successive touches that are at least two short touches or more applied in a reference time, a long touch, multi-touch, drag-and-drop, scroll-like, zoom-in, zoom-out, drag-type, and hover-like.
In addition, the photo received through the camera 121 is called preview, and it is a photo received in real time through the camera 121. Indeed, the photo (or the screen information contained in the photo) received via the camera 121 changes according to the displacement of the mobile terminal 100, the movement of an object (or a photographic subject) present in a viewing angle of the camera 121 or the like.
The photo (or preview) can be stored as a static image or as an animated image in the memory unit 170. The "photos received via the camera 121", which we will describe below. after, include a preview, a captured photo, a static image, namely an image that a control command places in a temporarily stationary state, and the like.
In addition, the mobile terminal performs an OCR function using the photo received through the camera 121. Specifically, the OCR function converts the screen information (an image) corresponding to a text portion included in the image into text. Photo. Hereinafter, in the light of the accompanying drawings, various functions performed by the photo capture function and the OCR function according to one embodiment of the present invention will be described.
In particular, Fig. 2 is a flowchart showing a control method according to an embodiment of the present invention, and Figs. 3A to 3C are conceptual diagrams representing the control method of Fig. 2. As we see in Figs. Figure 2, the controller 180 edits the received photo through the camera 121 on the display unit 151 (S210). For example, the controller 180 activates the camera 121 based on the selection (of the touch) of the icon of an application relating to the camera 121 or the icon of an application described in this specification, and edits the received photo through the activated camera 121 on the display unit 151. In addition, as we have seen previously, the photo is a preview.
On the other hand, when images corresponding to text portions having different characteristics are included in the photo, the controller 180 edits graphic objects in the photo which indicate that the images can be selected (S220). Specifically, when the controller 180 determines that images corresponding to the text portions having different characteristics are present in the photo received through the camera 121, the controller 180 edits graphic objects indicating that the images can be selected, close to the images (or so that the graphic objects are superimposed on the images).
When the user sets the camera 121 on an object (photographic subject) on which the text parts are printed (or handwritten), the images corresponding to the text portions are included in the picture received via the camera 121. In in addition, the image corresponding to the text portion has various characteristics (or attributes). In particular, the features described in this specification refer to references in which the image corresponding to the text portion is classified.
For example, these features are classified or defined by the fact that the text part is written in cursive script or that the text part is printed by a printing device (for example a printer) or is handwritten, or at least either the color of the text portion, the color superimposed on the text portion, or the color in the vicinity of the text portion, the shading of the text portion, the thickness of the text portion, the underlining of the text part, the inclination of the text part, the type of language and the like. In addition, text portions (or images corresponding to text portions) include a letter, an emoticon, a number, a symbol, and the like.
Further, the controller 180 determines whether or not the image corresponding to the text portion in the received photo is provided through the camera 121 or the characteristics of the image corresponding to the text portion using the OCR function or a photo analysis algorithm (specific algorithm that determines whether or not the image that corresponds to the text part).
When images corresponding to the text portions having different characteristics are included in the photo, the controller 180 edits the graphic objects on the photo, indicating that the images can be selected. In addition, the controller 180 may edit the graphic objects when the photo received through the camera 121 is a preview, and when a request to capture the photo is issued. For example, when the image received as a real-time view through the camera 121, as well as images corresponding to the text portions having different characteristics, is shown in the preview, the controller 180 edits the graphic objects on the preview.
As another example, when the request to capture the photo (the preview) is emitted (for example, by the touch of a photo capture button), the controller 180 captures the photo, and edits the captured photo. with graphic objects indicating that images corresponding to text parts with different characteristics are included in the captured photo. In addition, the captured photo may be a photo stored in memory 170 or a temporary static image temporarily stored. Now, we will describe an example in which graphic objects are edited when the request to capture the photo is issued. However, the description below can also be applied in a similar or identical manner to the preview (preview status). To go into details, as we see in Figure 3A (a), when the camera 121 is attached to a photographic subject in which textual elements are present, images 310a and 310b corresponding to the text portions are included in a picture. photo 300 received through the camera 121.
In addition, the images 310a and 310b have different characteristics. In particular, the image 310a corresponds to a text part having a first characteristic or a first attribute and the image 310b corresponds to a text part having a second characteristic or a second attribute different from the first characteristic or attribute. For example, image 310a is a handwritten text portion written by the user, and image 310b is a text portion printed by the printing device (eg, printer).
In this specification, for convenience, we describe an example in which images corresponding to text portions having two different characteristics are included in the photo. However, and without limitation, this specification also applies to images corresponding to text parts having three or more different characteristics. Further, when the images 310a and 310b corresponding to the text portions having different characteristics or attributes are included in the photo 300, the controller 180 edits graphic objects 320a and 320b (e.g., checkboxes ) indicating that the images 310a and 310b can be selected in the photo 300, as we see in Figure 3A (b).
As a first example, the graphic objects 320a and 320b are displayed in the preview state when the controller 180 recognizes that images corresponding to text portions having different characteristics are included in the received picture through the camera 121. As another example, the graphic objects 320a and 320b are edited from the touch of the photo capture button (or the receipt of a photo capture request), as we see in FIG. 3A ( at). In this case, a photo (in this case, a static image) resulting from the capture of the photo (preview) received by the camera 121 is edited on the display unit 151, and the graphic objects 320a and 320b are edited on the captured photo.
Returning to Fig. 2, when selecting an image from the images corresponding to the text portions having different characteristics, the controller 180 performs a predetermined function based on the characteristic or attribute of the selected image (S230). To go into details, the predetermined functions include different types of functions of using an image corresponding to a text portion. For example, the predetermined functions include limiting the editing of an image corresponding to a text portion, an OCR function of converting the text portion of the text image, the translation of a text portion after conversion. resulting from the conversion, performing a search by means of the text-after-conversion part, providing additional information relating to the text-after-conversion part, entering the text part after conversion into an application specific (for example, a memo application, a calendar application, or a schedule application) and the like.
More concretely, when one selects (by touching) the image (or the graphic object 320a edited on the image 310a), the control device 180 edits a graphic object associated with a first function in the photo. In addition, when selecting (by touching) the image (or the graphic object 320b), the controller 180 edits a graphic object associated with a second function different from the first function in the photo. For example, as we see in FIG. 3B (a), when selecting the image 310a (or the graphic object 320a), the control device 180 edits a graphic object 330a associated with the first function in the photo as we see in Figure 3B (b). For example, the first function limits the editing of the image 310a corresponding to the text portion having the first feature, for example by reducing the amount of information displayed in the image 310a.
As we see in FIG. 3B (b), when a touch is applied to a graphic object 330a associated with the first function, the editing of the image 310a is limited on the display unit 151. For example, image 310a is no longer displayed on display unit 141, as we see in FIG. 3B (c). In addition, after application of the touch, the control device 180 edits in the photo a graphic object 340a which is associated with a redissembling function of the limited edition image corresponding to the text portion.
To enter the details, when the graphic object 340a is touched, the control device 180 edits the image 310a corresponding to the limited edition text part in the photo. In addition, when the edition of the image 310a is limited, the control device 180 edits on the display unit 151 a storage icon with which a storage function is associated. When the storage icon is selected, the controller 180 saves the photo in the memory 170 in which the image corresponding to the text portion having the first feature is located.
As another example, illustrated in FIG. 3C (a), when selecting the image 310b (or the graphic object 320b), the control device 180 edits on the photo graphic objects 332b, 334b, 336b, and 338b which is associated with the second function, as we see in Figure 3C (b). For example, the second functions include various functions that can be performed by converting the text portion of the image to text and then using a text portion after conversion. For example, the second functions include at least one of the following functions: an internet search function, a sound transmission function, a translation function and a memo function.
As we see in FIG. 3C (b), the graphic objects with which the second function is associated, in this case a graphic object 332b with which the search function on the Internet is associated, a graphic object 334b with which the function is associated sound or audio emission, a graphic object 336b which is associated a memo function, and a graphic object 338b which is associated with the translation function, are displayed on the picture.
In addition, when a touch is applied to the graphic object (for example, the graphical object 334b with which the sound emission function is associated) with which the second function is associated, the control device 180 converts the text into text. image 310b corresponding to the text portion having the second characteristic, using the OCR function. Further, the controller 180 executes the second function (e.g., the sound emission function) associated with the affected graphic object 334b by using the text portion after conversion, as we see in Figure 3C (c). ).
Thus, there is provided a new user interface for performing a function that varies from one feature to another, when images corresponding to text parts with different characteristics are included in the picture. Further, there is described above a method in which a function is executed which varies according to whether the first and second characteristics are the handwritten text portion by the user or the text portion printed by the printing device. Embodiments will now be described in which the first and second characteristics are classified by color, with reference to the accompanying drawings.
In particular, Figs. 4A-4C are diagrams showing a control method according to another embodiment of the present invention. In this embodiment, the characteristics of the image corresponding to the text portion are classified according to the color of the image corresponding to the text portion included in the photo or the color (color superimposed on the image which corresponds to the text portion) of the neighborhood of the image corresponding to the text portion.
For example, as we see in Figure 4A (a), an image 400a corresponding to a text portion to which a first color is superimposed, an image 400b corresponding to a text portion to which is superimposed a second color different from the first color, and an image 400c corresponding to a text portion overlaid with any color (or overlaid with a third color) are included in the photo 300 received via the camera 121.
As we see in Figure 4A (b), the controller 180 selects an image corresponding to a text portion distinguished by its color. For example, the selection, as we see in Figures 2 to 3C (c), can be done on the basis of the touch of graphic objects (checkboxes) 410a, 410b, and 410c edited based on the recognition of images corresponding to the text parts which are distinguished by their colors. As another example, the selection can be done after editing the graphic objects 410a, 410b, and 410c once the request to capture the photo has been received and after applying the touch to the graphic objects (or images corresponding to the text parts).
As we see in Figure 4A (b), when selecting at least two of the images 400a, 400b, and 400c, the controller 180 edits the graphic object 420 associated with the memo function in the photo. As we see in Figure 4A (b), when the graphic object 420 is touched, the controller 180 converts the selected images 400a, 400b, and 400c into text using the OCR function. Further, as we see in Fig. 4A (c), the controller 180 executes the memo application, and displays text portions after conversion 440a, 440b, and 440c resulting from the conversion, on a display screen. execution 430 of the memo application.
Further, as we see in Figure 4A (c), the text-converted portions 440a, 440b, and 440c are displayed on the execution screen 430 of the memo application and are distinguishable from each other by their respective colors. For example, if we look at Figs. 4A (a) through 4A (c), the text conversion portion 440a is displayed on the run screen 430 so that the first color is superimposed. Likewise, the converted text part 440b is displayed on the execution screen 430 so that the second color is superimposed, and the text part after conversion 440c is displayed on the execution screen. 430 so that no color is superimposed.
In addition, the controller 180 displays a graphic object 450 in the execution screen 430 for editing a photo containing the image corresponding to the text portion. When selecting the graphic object 450, the control device 180 edits a picture shown in Fig. 4A (b) on the display unit 151. Thus, when an image corresponding to a text portion to which superimposes a predetermined color is included in a photo, the controller 180 extracts the image corresponding to the text portion to which the color is superimposed, and executes at least either a text conversion or a translation using the image extracted. In addition, the results (a text-after-conversion part and a text-translated part resulting from the translation) of the execution of text conversion and text translation are displayed on the execution screen of the memo application.
In addition, the predetermined color is a color other than colors in a specific color range (e.g., a range of blanks or a colorless range). Specifically, the controller 180 determines a color other than a specific color (a color in the white range) as the predetermined color. In addition, the predetermined colors include all the colors of a color gamut. For example, as we see in Figure 4B (a), a color chosen by the user is superimposed on a photographic subject which appears a text portion. When highlighting the text portion of the photographic subject with a highlighter, the text portion highlighted with the highlighter is considered a text portion to which a predetermined color is superimposed.
As we see in FIG. 4B (a), when at least two or more colors are superimposed on a text portion of the photographic subject, the images 460a and 460b corresponding to the text portions on which at least two predetermined colors are superimposed or more are included in the photo received through the camera 121, as we see in Figure 4B (b).
In this case, based on a user request (for example, when a touch is applied to the photo capture button or when a touch is applied to the graphic object associated with the memo function), the device control 180 extracts an image corresponding to a text portion to which the at least two predetermined colors are superimposed, and performs at least either a text conversion or a translation of the text using the extracted image.
Further, as we see in Fig. 4B (c), the controller 180 displays the results (the text-after-conversion part and the translation-translated text part) of the translation execution of the conversion. text and translation of the text, on the execution screen of the application memo, so that the results are distinguished by the at least two colors 470a and 470b. Indeed, when at least two colors are superimposed on an image corresponding to a text part included in the photo, the control device 180 displays (in) the text part after conversion and the text part after translation resulting from the translation on the execution screen of the application memo, so that the two parts forming text after conversion are distinguished by the at least two colors.
For example, the text-after-conversion portions resulting from the conversion and translation of the image corresponding to the text portion to which the first color is superimposed are displayed on the execution screen of the memo application of such that the first color is superimposed on the text portions after conversion, and the text-converted portions resulting from conversion and translation the image corresponding to the text portion to which the second color is superimposed are displayed on the execution screen of the memo application so that the second color is superimposed on the text parts after conversion. Thus, thanks to this configuration, one has an application (or a user interface) which makes it possible to optimally create a list of vocabulary.
In addition, as we see in Figure 4C (a), when the user sets the camera 121 on a calendar (photographic subject), a graphic image corresponding to the calendar is included in a picture received through the camera 121 When the graphic image corresponding to the calendar is included in the picture, the controller 180 extracts an image 480a corresponding to a text portion included in the graphic image. When the photo capture request is received, or the image corresponding to the text portion is recognized by the controller 180, the extraction takes place.
Further, the controller 180 converts the extracted image into text, and as we see in FIG. 4C (b), displays (in) a text portion after conversion 480b resulting from the conversion to (in) a screen 490 of a calendar application. Further, the conversion converting text portion 480b resulting from the conversion, illustrated in FIG. 4C (b), is displayed on the execution screen 490 of the calendar application such that the text portion after 480b conversion corresponds to the color in which the 480 image is displayed, and so that the 480b converted text portion corresponds to the position (date) at which the 300 photo is displayed.
For example, the controller 180 displays or enters a circular mark and a text portion after conversion ("meeting") on the date of "14" of the calendar application run screen, using a a graphic image corresponding to the calendar included in the photo received via the camera 121, such that the circular mark and the text-converted part correspond to a circular mark drawn by hand on the 14th of the calendar (photographic subject) and at a meeting ("meeting") written by hand the "14" of the calendar, respectively.
As another example, the control device 180 displays or between a text part after conversion "d-day" resulting from the conversion of the mention "d-day" handwritten the "22" of the calendar (photographic subject) and to which superimposes a specific color, the "22" on the run screen of the calendar application, so that the specific color is superimposed on "22" on the run screen of the calendar application. Thus, thanks to this configuration, there is a user interface through which it is possible to optimally enter a handwritten text portion on the calendar.
We have just described a method in which various functions are performed using features that are distinguished by colors. We will now describe, with reference to the accompanying drawings, an embodiment in which a characteristic corresponding to the thickness of a character is used. In particular, Figs. 5A-8 are conceptual diagrams representing a control method according to another embodiment.
Let us examine FIGS. 5A to 5F, which, by way of example, describes a menu constituting a photographic subject. As seen in Fig. 5A (a), the photo 300 received through the camera 121 includes an image 500a (e.g., an image corresponding to a text portion having a thickness of a predetermined value or greater, or an image corresponding to bold characters) corresponding to a text portion having the first feature, and an image 500b (for example, an image corresponding to a text portion having a thickness less than the predetermined value, or an image that is not not corresponding to the bold type) corresponding to a text part having the second characteristic.
Fig. 5A (b) shows highlighted images 510a and indicating image portions containing text. Further, the controller 180 can select only an image corresponding to a text portion having any characteristic (e.g., the first characteristic) of the first and second characteristics, from the photo received through the camera. The selection is made on the basis of the reception of the request for photo capture or the recognition of the first and second characteristics.
When selecting the image corresponding to the text portion having either of the first and second features (or when the image having either of the first and second features is recognized by the device 180), the controller 180 edits graphic objects 520 associated with a predetermined function in the photo. When selecting any graphics object 520, the controller 180 converts the selected image into text and executes a function (e.g., the Internet search function, the translation function, or the like) associated with the graphic object. selected, using the text portion after conversion.
Further, as shown in Figures 5B (a) and 5B (b), the controller 180 selects only images 512a, 514a, 516a, and 518a corresponding to a text portion having one or other first and second characteristics from a first photo 300a received through the camera 121. Also, based on the user request, i.e. when applying a touch 530 to any of the images corresponding to text portions included in a second picture 300b different from the first picture 300a, the control device 180 edits pictures 512b, 514b, 516b, and 518b relating to the pictures selected on the picture unit. 151, based on the affected image and the selected images.
For example, when only images 512a, 514a, 516a, and 518a corresponding to the text portions having any of the features (e.g., the first feature) are selected (or recognized) from the first photo 300a, the control device 180 edits a graphic object 520a associated with a reception function of a new second photo 300b, as we see in FIG. 5B (a). The user request is the touch of the graphic object 520a. In addition, the controller 180 converts the image corresponding to the text portion having any of the features included in the first photo 300a to text and (temporarily) saves a text portion after conversion to the memory 170. In furthermore, when the graphic object 520a is touched, the controller 180 receives different pictures (the first picture) more than once, and selects only an image corresponding to the text part having any of the features that are included in each of the photos (each of the first photos). In addition, the controller 180 converts the selected image into each of the photos into text, and records a text portion after conversion in the memory 170.
In the example of Figure 5B, a text portion having any of the features corresponds to a meal menu. Further, based on the selection of the graphic object 520a, the controller 180 edits the second photo 300b to the display unit 151. As we see in Fig. 5B (b), when When touching any image 530 of the images corresponding to the text portions included in the second photo 300b, the controller 180 converts the affected image into text. For example, the affected image is any of the text parts included in a wine list. Further, the controller 180 extracts (selects) the best-described recommended meal from the affected image (or text-converted portion) from the image corresponding to the text portion having any of the features included in the previously captured photo. The recommended meal best described by the affected image may have already been stored on an external server (which is connected through the Internet) or in memory 170.
Furthermore, as we see in Figure 5B (b), on the basis of the affected image and the image corresponding to the text portion having any of the features (i.e. the base of the images that are selected from the first picture), the controller 180 edits the images (i.e., a photo of the meal menu) 512b, 514b, 516b, and 518b that relate to the images selected on the display unit 151.
For example, when the meals that are best described by the affected image (a wine list) are images corresponding to meal 1, meal 2, meal 3 and meal 4, among the images corresponding to the text portions selected from the first picture, the control device 180 receives pictures of dishes corresponding to menu 1, menu 2, menu 3 and menu 4, of the external server, and edits the received photos 512b, 514b, 516b, and 518b of the different dishes on the display unit 151. Thus, there is a user interface through which one easily receives, through the camera, information on a recommended dish.
In addition, the controller 180 edits additional information on the display unit 151 (or photo) using an image corresponding to a text portion. For example, as shown in Fig. 5C, the controller 180 selects only the images 512a, 514a, 516a, and 518a corresponding to the text portion having any characteristic (e.g., the first feature) of the first and second characteristics from the photo 300 received via the camera 121.
In addition, the controller 180 edits specific graphic objects (e.g., check boxes) 512c and 518c in the vicinity of the images 512a and 518a on which additional information can be output from the selected images. In addition, the controller 180 converts the selected images into text and receives additional information by searching the Internet or the like, using a text portion after conversion. Further, based on the text-after-conversion portion resulting from receiving and converting the additional information, the controller 180 edits a specific graphic object in the vicinity of the image, corresponding to the text portion.
On the other hand, when at least one graphic object (for example, the graphic object 512c) is selected from among specific graphic objects, the control device 180 edits complementary information 540a relating to the image 512a corresponding to the graphic object. specific selected, in the photo. In addition, whether the additional information is edited or not, all the specific graphic objects are displayed near the image corresponding to the text portion having any of the features that is included in the photo. The user can also directly touch the image 512a to edit the additional information 540a rather than touching the check box 512c.
Furthermore, as we see in Figure 5D, when selecting at least one or more specific graphic objects (for example, multiple specific graphic objects), the controller 180 may edit additional information (e.g. tariff information 550) relating to the selected specific graphic objects, on the display unit 151.
In addition, the mobile terminal according to one embodiment of the present invention can perform various functions. For example, as we see in Figure 5E (a), the controller 180 selects only an image corresponding to a text portion having either of the first and second characteristics from the photo received by The controller 180 also converts the selected image into text, and displays a text portion after conversion so that the text portion after conversion is superimposed on the selected images.
Furthermore, when a graphic object 528 associated with the translation function is touched at the time of selecting the image, the controller 180 converts the selected images into text, translates a text portion after conversion, and then displays on the photo a text forming part after conversion resulting from the translation, as we see in Figure 5E (b). Further, the conversion text portions 512d, 514d, 516d, and 518d resulting from the translation are superimposed on the images corresponding to the text portions.
In addition, a graphic object 560 by which the selected image can be adjusted is displayed on the images corresponding to the text portions having any of the features. As seen in Figs. 5E (a) through 5E (d), the controller 180 modifies (adjusts) the selected images based on the drag-and-drop type of touch applied to the graphic object 560. More specifically, this operation consists of adjusting a section defined block by block.
In addition, when converting the image corresponding to the text portion to text or translating a text portion after conversion, the graphic object 560 is displayed in the photo. As we see in Figure 5E (c), when selecting any one (for example, a text-forming part after translation 512d resulting from the translation) of the text-forming parts after translation and then touching an object 522 associated with the search function on the Internet, the controller 180 edits a web page 570 resulting from the search performed using the text-translation portion 512d on the display unit 151, as we see on the Figure 5E (d).
The search function applies in an identical or similar manner, not only to the text part after translation but also to the text part after conversion. Furthermore, as we see in Figure 5F (a), when only the images 512a, 514a, 516a, and 518a corresponding to the text portions having either of the first and second characteristics are selected by the device 180, and when touching a graphic object 524 associated with an SNS function, the control device 180 edits an execution screen 590 of an SNS application on the display unit 151.
Further, as seen in Figs. 5F (a) and (b), the controller 180 records a received photo 500 through the camera 121 as an attached file SNS 580. In addition, the device command 180 converts images 512a, 514a, 516a, and 518a corresponding to text portions that are selected, and automatically enters the text portions after conversion 512f, 514f, 516f, and 518f resulting from the conversion, into an input window from run screen 590.
In addition, the conversion text portions 512f, 514f, 516f, and 518f resulting from the conversion can be entered into the input window without any modification, and entered into the input window with a specific symbol that has just been received. add to the text-converting portions 512f, 514f, 516f, and 518f. As seen in FIG. 5G (a), when selecting an image corresponding to a restaurant from images corresponding to text portions included in the photo received through the camera 121, the controller 180 edits a graphic object 513 associated with a function of editing images of dishes provided by the restaurant, in the photo.
More concretely, once the image corresponding to the text portion having either of the first and second features has been selected and a specific image selected on the basis of the user request, the The controller 180 converts the specific image into text, and from a text portion after conversion resulting from the conversion, determines whether or not the selected specific image is an image corresponding to the name of the restaurant. In addition, the controller 180 searches the Internet using the text portion after conversion or determines the name of the restaurant based on the data stored in the memory 170. In this case, the controller 180 edits the data. graphic object 513 on the photo.
When selecting the graphic object 513, the controller 180 searches for images of dishes that may be offered by the restaurant, using the text-converted portion, and selects at least a few images of dishes 592 resulting from the search. . For example, at least some dishes images that are edited in the photo are a predetermined number of images of dishes selected in descending order of the number of times the dish is recommended or in descending order of the number of times where the dish is searched. Thanks to this configuration, we have a user interface thanks to which we can obtain additional information optimally using the camera.
We will now describe an embodiment of the present invention in more detail with reference to the accompanying drawings. As we see in Fig. 6A (a), when the user sets the camera 121 on a label attached to a garment or the like, an image 600 corresponding to a symbol drawn on the label is included in the photo 300 received by through the camera 121. In response to the photo capture request, the controller 180 recognizes the image 600 corresponding to the symbol drawn on the label, and as we see in Figure 6A (b), edits additional information 610 on the photo. In addition, as explained above, a specific graphic box (a checkbox) may be displayed near the symbols, and additional information may be provided on the image 600 corresponding to the symbol. In addition, when selecting the specific graphic object, the controller 180 only edits the complementary information corresponding to the symbol of the specific graphic object selected, on the display unit 151.
In another example, shown in Fig. 6B (a), a graphic image 620 corresponding to an invoice is included in the photo 300 received via the camera 121. When the controller 180 determines that the graphic image 620 is a photographic subject relating to a transfer between accounts, as an invoice, the controller 180 edits an icon 630 of a banking application on the photo, as we see in Figure 6B (b). In addition, the 630 icon is edited after receiving the photo capture request.
Further, the controller 180 recognizes an image corresponding to a text portion included in the graphic image 620, and edits specific objects 622a and 622b in the vicinity of the image corresponding to the specific information (e.g. account or similar) in the recognized image.
When selecting any one (e.g., the specific graphical object 622a) of the specific graphical objects 622a and 622b, and when selecting the icon 630, the controller 180 edits an execution of the banking application. corresponding to the icon selected on the display unit 151. In addition, the controller 180 converts the image 622a corresponding to the text portion that has been selected into text.
The controller 180 also automatically enters the text portion after conversion (e.g., the account number) resulting from the conversion into a specific portion (e.g., an account number entry box) included in the screen. execution of the banking application. In addition, the controller 180 recognizes an image corresponding to information on an amount, and converts the recognized image into text.
In addition, the controller 180 automatically displays (enters) a text portion after conversion (amount information) resulting from the conversion into an amount entry box included in the application execution screen. banking. With this configuration, there is a user interface that allows you to make a bank transfer in a more convenient way.
In addition, various complementary information is provided by means of an image corresponding to a text portion included in a photo received through the camera 121. For example, as we see in Fig. 7A (a), an image comprising a text portion relating to a recipe is included in a photo received from the camera 121. In this case, when the photo capture request is received, the controller 180 recognizes corresponding images 710a, 710b, 710c, and 710d. to the recipe text portions included in the photo, and as we see in Figure 7A (b), selects the recognized images.
Further, when the controller 180 determines that the image corresponding to the text portion included in the picture relates to the recipe, the controller 180 displays graphic objects 720a and 720b associated with a predetermined function relating to the recipe, as we see in Figure 7A (b). For example, the predetermined function changes an amount relative to the recipe or modifies units.
As we see in Figure 7A (b), when a graphic object 720a is associated with the quantity modification function, the controller 180 edits on the display unit 151 a pop-up window (or window). 730a by which the quantity is selected, as we see in Figure 7A (c). In addition, the amount (e.g., "for 4 people") corresponding to the image 710a of the text portion included in the photo does not appear in the selection window.
In addition, when a specific quantity is selected, the controller 180 displays quantity information 740a and 740b as a result of changing the product quantities according to the selected quantity, in the vicinity of the corresponding portion image 710b. forming text relating to a quantity of product included in the photo, as we see in Figure 7A (d).
As we see in Fig. 7B (b), when selecting a graphic object 720b associated with a unit modification function, the controller 180 edits a new picture 300b on the display unit 151, as we see it in Figure 7B (c). In addition, when the user sets the camera 121 on an object (for example, a cup) to measure a quantity, an image corresponding to the object is displayed on the new photo 300b.
In addition, the controller 180 determines the distance between the main body and the object using multiple cameras 121a and 121b or 121c and 121d shown in Figures 1B and 1C and, depending on the distance, calculates the volume of the camera. 'object. In addition, the control device 180 displays information 310b corresponding to the volume calculated in the photo. When selecting a button 320b associated with the unit modification (or unit measurement) function and displayed on the new picture 300b, the controller 180 displays the initial picture 300a on the display unit 151, and displays information 752a and 752b resulting from the modification of the units (or measurement of the units) according to the calculated volume of the object in the vicinity of the images 750a and 750b.
More precisely, the control device 180 modifies the information relating to a volume included in the photo according to a volume corresponding to an object for which a new photo has been captured, and edits the information modified on the photo. For example, when the quantity of the product included in the photo is half a cup with a capacity of 100 ml, the quantity of product corresponds to a quantity of 50 ml. When the calculated volume is 150 ml, the modified information displayed near the image corresponding to the text portion relating to the quantity of product is "one-third of a cup with a capacity of 150 ml" . Thanks to this configuration, we have a user interface that we optimize for a recipe using the camera.
Further, as we see in FIG. 8 (a), a graphic image 800 corresponding to a musical score is included in the photo 300 received via the camera 121. In this case, the control device 180 analyzes the image. 800 graphic image, determines the musical instrument to which the partition corresponds, and as we see in Figure 8 (b), displays a graphic object 820a indicating the corresponding instrument on the picture. In addition, the control device 180 edits graphic objects 810a and 810b associated with a function relating to the musical score in the photo.
The graphic objects 820a, 810a, and 810b, illustrated in FIG. 8 (a), are edited after receiving the photo capture request. The functions relating to the score include changing the musical instrument (or musical score), playing a musical instrument or the like. Furthermore, as we see in Fig. 8 (b), when selecting the graphic object 810b associated with the musical instrument changing function, the controller 180 edits a pop-up window (or window). selection) 830 by which another musical instrument is selected to play on the display unit 151, as we see in Figure 8 (c).
When selecting a musical instrument other than the particular musical instrument, as we see in FIG. 8 (d), the controller 180 transforms the graphic image 800 corresponding to the score into an image of a musical instrument. 840 musical score corresponding to the other musical instrument. An 820b image indicating the other musical instrument can also be displayed on the 840 partition.
As we see in Fig. 8 (d), when selecting the graphic object 810a associated with the function of playing a musical instrument, based on the graphic image 840 (the image of the partition) edited on the display unit 151, the controller 180 edits a sound resulting from the reading of the graphic image 840, as its obtained by playing the musical instrument 820b associated with the image graph 840. With this configuration, various functions are performed in different areas using a photo received through the camera.
As previously explained, according to one embodiment of the present invention, various functions may be performed based on characteristics of an image corresponding to a text portion included in a photo. In addition, there is a new user interface by which an image corresponding to a text part included in a picture can be converted into text, and the text part after translation resulting from translation can thus be entered into a specific application. , indicating a characteristic of the image corresponding to the text portion. In addition, there is a user interface through which further optimized information can be obtained by using a picture received via the camera.
Various embodiments may be implemented by means of a machine-readable medium on which instructions to be executed by a processor are stored to perform various methods presented herein. Examples of possible machine-readable media include a hard disk, SSD, SDD (hard disk on silicon), read-only memory, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, other types of storage media presented herein, and combinations of these different media. Where appropriate, the machine-readable medium may be embodied as a carrier wave (for example, a transmission wave over the Internet). The processor may include the controller 180 of the mobile terminal.
The aforementioned embodiments and advantages are of a purely representative nature and should not be construed as limiting the present invention. The present teachings can easily be applied to other types of apparatus. This description is intended to serve as an illustration, not to limit the scope of the claims. Those skilled in the art will see that many possibilities, modifications and variants can be envisaged. The features, structures, methods, and other features of the representative embodiments described herein may be combined in different ways to provide additional and / or alternative representative embodiments.
Just as the present attributes can be implemented in several forms without departing from their characteristics, it should also be pointed out that the embodiments described above are not limited by any of the details of the foregoing description. unless otherwise indicated, but shall be considered in a broad sense within their scope as defined in the appended claims, and therefore, any changes and modifications that fall within the ins and outs of the claims, or Equivalents of these ins and outs, are therefore intended to be encompassed in the appended claims.

权利要求:
Claims (15)
[1" id="c-fr-0001]
A mobile terminal comprising: a camera (121); a touch screen (151) for editing a received photo via the camera (121), said photo including at least a first image text portion having a first attribute and a second image text portion; with a second attribute different from the first attribute; and a controller (180) adapted to: display first and second graphic objects indicating that the first and second image text portions can be selected, perform a first predetermined function corresponding to the first attribute of the first text portion of the first portion; image when selecting the first graphic object or the first image text part, and executing a second predetermined function corresponding to the second attribute of the second image text part when selecting the second graphic object or the second part forming image text.
[2" id="c-fr-0002]
The mobile terminal of claim 1, wherein the first and second graphic objects comprise at least one of the selectable check boxes, or dotted lines around the respective first and second image text portions, or the highlighting. first and second respective image text portions.
[3" id="c-fr-0003]
Mobile terminal according to any one of claims 1 and 2, wherein the first predetermined function corresponds to a reduction or limitation of the display of the first image text portion on the touch screen (151), and in wherein the second predetermined function is the conversion of the second image text portion of the touch screen (151) to text and the use of the converted text in a specific application.
[4" id="c-fr-0004]
A mobile terminal according to any one of claims 1 to 3, wherein the controller (180) is further adapted to: when selecting the first graphic object or the first image text portion, performing the first function predetermined manner limiting the output of the first image text portion and displaying another graphical object to redisplay the first image text portion on the touch screen (151), and when selecting the second graphical object or the second image text portion, executing the second predetermined function converting the second image text portion of the touch screen (151) to text and using the converted text in the specific application comprising at least one of a text search function. Internet, either a sound transmission function, a translation function, or a memo function.
[5" id="c-fr-0005]
The mobile terminal of claim 4, wherein the controller (180) is further adapted to display selectable application icons corresponding respectively to the Internet search function, the audio emission function. , the translation function and the memo function.
[6" id="c-fr-0006]
The mobile terminal according to any one of claims 1 to 5, wherein the first attribute of the first image text portion corresponds to a handwritten text and the second attribute of the second image text portion corresponds to a printed text. .
[7" id="c-fr-0007]
The mobile terminal according to any one of claims 1 to 6, wherein the first attribute of the first image text portion corresponds to a first color and the second attribute of the second image text portion corresponds to a second color. .
[8" id="c-fr-0008]
The mobile terminal of claim 7, wherein the controller (180) is further adapted to: display a memo graphical object for executing a memo application when selecting the first and second graphical objects or the first and second image text portions, when selecting the graphic memo object, converting the first image text portion to a first text and the second image text portion to a second text, and displaying the first text and the second text on a run screen of the memo application, and wherein the first text is displayed with the first color and the second text is displayed with the second color.
[9" id="c-fr-0009]
Mobile terminal according to any one of claims 7 and 8, wherein the first and second colors are superimposed by a user, wherein the first predetermined function corresponds to the conversion of the first image text portion into a first one. text, the translation of the first text converted into a first different text from another language, and the display of the first text and the first different text, and wherein the second predetermined function corresponds to the conversion of the second part forming an image text into a second text, translating the second text converted into a second different text from another language, and displaying the second text and the second different text.
[10" id="c-fr-0010]
The mobile terminal of claim 9, wherein the first text and the first different text are displayed with the first color, and the second text and the second different text are displayed with the second color.
[11" id="c-fr-0011]
The mobile terminal of any one of claims 1 to 10, wherein the photograph comprises a calendar image, and wherein the controller (180) is further adapted to: convert the handwritten image text portions. from the calendar image to text, and display the converted text on the same date as the handwritten image text portions on a run screen of a calendar application.
[12" id="c-fr-0012]
The mobile terminal of claim 11, wherein the converted text is displayed in a color corresponding to the color of the handwritten image text portions of the calendar image.
[13" id="c-fr-0013]
The mobile terminal of any one of claims 1 to 12, wherein the controller (180) is further adapted to: display another received photo through the camera (121), and when selecting a text image portion in the other photo, displaying images corresponding to a link between the text included in the text image portion of the other photo and the text included in the first and second image text portions .
[14" id="c-fr-0014]
The mobile terminal of any one of claims 1 to 13, wherein the controller (180) is further adapted to: display a graphic object in proximity to the first image text portion and the second portion forming image text indicating that additional information is available, and displaying the additional information on the photo when selecting the graphic object.
[15" id="c-fr-0015]
15. A method of controlling a mobile terminal, the method comprising: editing, via a touch screen (151), a photo received through the camera (121), said photo including at least a first image text portion having a first attribute and a second image text portion having a second attribute different from the first attribute; displaying, via the touch screen (151), first and second graphic objects indicating that the first and second image text portions can be selected; executing, via a controller (180), a first predetermined function corresponding to the first attribute of the first image text portion when selecting the first graphic object or the first forming part. picture text; and executing, via the controller (180), a second predetermined function corresponding to the second attribute of the second image text portion when selecting the second graphic object or the second text portion. image.

类似技术:

---

公开号 | 公开日 | 专利标题

---

FR3041785A1|2017-03-31|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3021133B1|2019-08-30|MOBILE TERMINAL AND METHOD FOR CONTROLLING THE MOBILE TERMINAL

---

FR3022368B1|2019-06-21|WATCH-TYPE TERMINAL AND CONTROL METHOD THEREOF

---

FR3021136A1|2015-11-20|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3031601B1|2019-08-30|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3021424B1|2019-09-20|MOBILE TERMINAL AND METHOD FOR CONTROLLING THE MOBILE TERMINAL

---

FR3021766A1|2015-12-04|MOBILE TERMINAL AND METHOD FOR CONTROLLING THE MOBILE TERMINAL

---

FR3026201A1|2016-03-25|

---

FR3022367A1|2015-12-18|

---

FR3022649A1|2015-12-25|

---

FR3024786A1|2016-02-12|MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME

---

FR3025328B1|2019-07-12|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3043478A1|2017-05-12|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3019665A1|2015-10-09|

---

FR3039673A1|2017-02-03|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3021134A1|2015-11-20|MOBILE TERMINAL

---

FR3021485A1|2015-11-27|MOBILE DEVICE AND METHOD OF CONTROLLING THE SAME

---

FR3021425A1|2015-11-27|

---

US9565354B2|2017-02-07|Mobile terminal for refocusing of a captured image and method for controlling the same

---

FR3039674A1|2017-02-03|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

FR3046470B1|2019-11-08|MOBILE TERMINAL

---

FR3021767A1|2015-12-04|MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME

---

FR3022648A1|2015-12-25|

---

FR3042084B1|2019-11-08|MOBILE TERMINAL AND METHOD OF CONTROLLING THE SAME

---

CN106488002B|2019-07-23|Mobile terminal and the method for controlling mobile terminal

同族专利:

---

公开号 | 公开日

---

EP3147816A3|2017-06-14|

---

EP3147816A2|2017-03-29|

---

FR3041785B1|2019-12-20|

---

US20170090693A1|2017-03-30|

---

CN106557226A|2017-04-05|

---

KR20170037424A|2017-04-04|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

US6181255B1|1997-02-27|2001-01-30|The Chamberlain Group, Inc.|Multi-frequency radio frequency transmitter with code learning capability|

---

US6940617B2|2001-02-09|2005-09-06|Matsushita Electric Industrial Co., Ltd.|Printing control interface system and method with handwriting discrimination capability|

---

JP2006018522A|2004-06-30|2006-01-19|Canon Inc|Image edition system, method, and program|

---

US20120290601A1|2007-11-15|2012-11-15|Master Wave International Co., Ltd.|Image-based Data Management Method and System|

---

KR101537598B1|2008-10-20|2015-07-20|엘지전자 주식회사|Mobile terminal with an image projector and method for controlling the same|

---

US20120083294A1|2010-09-30|2012-04-05|Apple Inc.|Integrated image detection and contextual commands|

---

EP2439676A1|2010-10-08|2012-04-11|Research in Motion Limited|System and method for displaying text in augmented reality|

---

EP2801065A4|2012-01-05|2015-08-05|Visa Int Service Ass|Transaction visual capturing apparatuses, methods and systems|

---

KR101860341B1|2011-09-09|2018-05-24|엘지전자 주식회사|Mobile terminal and control method for the same|

---

KR102147935B1|2013-08-29|2020-08-25|삼성전자주식회사|Method for processing data and an electronic device thereof|US20180033175A1|2016-07-28|2018-02-01|Sharp Kabushiki Kaisha|Image display device and image display system|

---

KR20190101731A|2018-02-23|2019-09-02|삼성전자주식회사|Method for displaying text information on an object contained in an image by compensating for motion generated during time of receiving text information from an external electronic device and electronic device thereof|

---

US10817677B2|2018-08-28|2020-10-27|Read Twogether Ltd|Single-line display of composite-text|

---

US10346549B1|2018-08-28|2019-07-09|Read TwoGether Ltd.|Single-line display of composite-text|

---

USD927462S1|2018-11-09|2021-08-10|Sony Corporation|Audio system|

---

CN109766978B|2019-01-17|2020-06-16|北京悦时网络科技发展有限公司|Word code generation method, word code identification device and storage medium|

---

KR20200100918A|2019-02-19|2020-08-27|삼성전자주식회사|Electronic device for providing various functions through application using a camera and operating method thereof|

---

US11263557B2|2019-09-11|2022-03-01|REQpay Inc.|Construction management method, system, computer readable medium, computer architecture, computer-implemented instructions, input-processing-output, graphical user interfaces, databases and file management|

---

USD930610S1|2019-09-29|2021-09-14|Sony Corporation|Speaker|

---

US10956016B1|2019-11-14|2021-03-23|Rockwell Collins, Inc.|No look touchscreen panel / CDU|

---

US11093691B1|2020-02-14|2021-08-17|Capital One Services, Llc|System and method for establishing an interactive communication session|

---

CN112183122A|2020-10-22|2021-01-05|腾讯科技（深圳）有限公司|Character recognition method and device, storage medium and electronic equipment|

法律状态:
2017-05-30| PLFP| Fee payment|Year of fee payment: 2 |

---

2017-07-21| PLSC| Publication of the preliminary search report|Effective date: 20170721 |

---

2018-05-29| PLFP| Fee payment|Year of fee payment: 3 |

---

2019-03-27| PLFP| Fee payment|Year of fee payment: 4 |

---

2020-04-24| PLFP| Fee payment|Year of fee payment: 5 |

---

2022-02-11| ST| Notification of lapse|Effective date: 20220105 |

优先权:

---

申请号 | 申请日 | 专利标题

---

KR1020150137030|2015-09-25|

---

KR1020150137030A|KR20170037424A|2015-09-25|2015-09-25|Mobile terminal and method for controlling the same|

---